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SUMMARY 


An  Identification  Point  Moael  was  constructed  to  assist  Air  Force 
planners  in  preaicting  the  effects  of  stress  upon  aircraft  maintenance  time. 
Relevant  causal  factors  that  would  result  in  an  increase  or  decrease  in 
maintenance  t-ime  were  i^er,*- prototype  of  a  predict!.^. .  Lo.^1  was 
oescribed  that  utilizes  these  factors. 

The  use  of  this  tool  in  two  modest  studies  yielded  41  preaictions  for 
the  situation  in  which  adequate  preparations  could  be  made  for  the  hazardous 
situation.  Three  more  data  points  were  collected  for  a  situation  in  which 
preparation  could  not  be  made. 

Suggestions  were  made  about  enhancement  and  further  use  of  the 
Identification  Point  Model. 
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PREFACE 

The  work  reported  in  this  technical  paper  was  performed  by  Klein 
Associates  under  subcontract  from  RJO  Enterprises,  Inc.,  Dayton,  Ohio.  The 
work  was  accomplished  under  ASD  contract  F33657-84-D-0315-0011/PTD 
0315-RL02-10-03  for  the  Air  Force  Human  Resources  Laboratory  (AFHRL) . 

Ms.  Cheryl  L.  Batchelor  monitoaeo  this  subcontract  for  the  Laboratory. 

This  effort  is  a  portion  of  the  AFHRL  Maintenance  and  Combat  Support 
thrust  ana  the  Combat  Logistics  Systems  subthrust,  and  is  specifically  a  part 
of  the  Maintenance  Reaainess  task.  It  supports  the  Laboratory's  interest  in 
the  impact  of  combat  stress  on  the  capabilities  of  a  maintenance 
organization.  This  is  a  follow-on  effort  to  further  develop  a  methodology 
developed  by  Klein  Associates  to  quantify  the  impact  on  stress  on  the 
performance  of  technical  tasks. 

Klein  Associates  acknowledges  the  valuable  assistance  of  Chief  Randy 
Staley,  David  Looney  and  Tim  Parks  of  the  Washington  Township  Fire  Department 
of  Centerville,  OH  for  providing  insight  into  their  sometimes  stressful 
environment.  The  support  provided  by  these  individuals  permitted  access  to 
valia  data  sources  and,  therefore,  contributed  immensely  to  the  quality  of 
this  jjaper.  Klein  Associates  also  wishes  to  thank  Mr.  Marvin  Thordsen  for  his 
assistance  in  collecting  data  and  Ms.  Beth  Crandall  for  her  critical  review  of 
earlier  drafts  of  this  paper. 
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Preaiction  Model  for  Estimating  Performance  Impacts 
of  Maintenance  Stress 


1.  INTRODUCTION 

■i 

Future  comoat  conditions  ma>  expose  maintenance  technicians  to  hazards 
such  as  bombs,  bullets,  and  nucleat/biological/chemical  (NBC)  warfare  condi¬ 
tions.  The  speed  ana  manner  of  aircraft  maintenance  task  performance  under 
these  conditions  are  critical  aeterminants  of  how  fast  aircraft  can  be 
returncu  to  combat.  The  speed  at  which  aircraft  can  be  turned  around  during 
normal  operations  ana  during  simulatea  surge  conditions  can  be  studiea  and  the 
oata  used  to  calculate  sortie  rates.  ik)wovef7 — sueh-otudy-methods  de  -not'  — 

performing  mai.nt«nanee  ins-'  life-threatening 
>i..««vi torments.  The  study  of  performance  of  even  routine  maintenance  tasks  in 
this  t j pe  of  environment  must  take  into  account  the  effects  of  psychological 
stress. 

because  the  effects  of  psychological  stress  upon  maintenance  personnel  can 
have  a  critical  impact  upon  the  time  needed  to  return  aircraft  to  battle,  the 
effect  of  such  stress  must  be  addressed^  Unless  the  real  and/or  perceived 
stress  upon  maintenance  persojjnei'’'is  accounted  for,  this  important  variable 
may  upset  all  calculations  ^^bout  sortie  rates  and  negate  the  effectiveness  of 
many  f>'/rmal  planning  efforts?^/ ) 

'  T -  " 

The  purpose  of  this  paper  is  to  describe  the  prototype  of  a  tool  intended 
to  assist  the  Air  Force  in  predicting  the  effects  of  psychological  stress  upon 
maintenance  timie. 

In  the  first  phase  of  this  effort,  data  were  collected  from  experienced 
maintenance  personnel  in  hazardous  chemical  plants  and  mM-.intenance  officers 
from  the  Air  National  Guard  to  estimate  and  predict  maintenance  time  in  Air 
Force  combat  conditions.  Tne  Comparison  Based  Prediction  (CBP)  method  was 
used  to  structure  and  elicit  expert  judgments  and  predictions  about 
mainte.iance  time.  It  was  found  that  maintenance  tasks  generally  were 
predicted  to  take  20%  longer  in  stressful  conditions.  Causal  factors  that 
were  :  -levant  to  the  performance  of  technical  tasks  undei  stressful  conditions 
were  identified.  A  follow-on  progect  used  this  information  (and  collected 
some  additional  data)  to  moael  the  effects  of  stress  upon  maintenance  time. 

The  Coiiiparison  b ^ I’rcuictio.’i  Mr  thod 

The  CbP  rnetiiOj  cas  aeveloptu  ty  Klein  Associates  as  a  prediction  method. 
CbP  has  proved  to  be  successful  iii  situations  characterized  by  ambiguous  or 
missing  uata  or  uncertainty  about  critical  elements  of  ttiC  prediction  scenario 
(John,  Klein,  &  Taynor,  1986;  John,  Strobhar,  &  Klein,  lubb;  Klein,  19b2; 

Klein  &  John,  1985;  Klein  &  John,  1986;  Klein  &  Wei t zen! c id ,  l9oz;  Klein  & 
Williams,  1983).  Formally,  it  is  a  system  of  reasoning  oy  analogy,  predicting 


to  an  unknown  case  by  drawing  upon  what  is  known  about  a  iamiliar,  comparable 
case,  ompiricaily,  it  is  a  means  of  significantly  increasing  the  validity  and 
reliability  of  very  difficult  predictions.  Operationally,  it  is  a  way  to  make 
predictions  in  situations  in  which  other  methoas  ao  not  appear  useful. 

In  obtaining  preaictions,  the  CBP  methou  capitalizes  upon  the  natural 
human  analogical  reasoning  process.  Most  typically,  the  predictions  are  made 
by  experts  in  the  content  domain  of  interest.  The  process  by  which  they  make 
their  predictions  is  structured  anc  probed  so  that  the  causal  factors  most 
salient  in  both  the  analog  and  the  problem  at  hand  are  highlighted  and 
documented.  Because  the  causal  factors  drive  the  predictions,  they  are 
documented  such  that  the  reasoning  employed  by  the  experts  can  be  made 
explicit  to  others.  If  errors  have  been  maae,  or  an  inappropriate  analog 
chosen,  this  also  is  made  apparent  by  the  audit  trail  of  how  and  why  the 
prediction  was  made. 

Previously,  the  methoo  has  been  used  in  a  face-to-tace  interview  format. 
Currently,  under  sponsorship  of  the  U.S.  Air  Force  Weapons  Laboratory  at 
Kirtlar.d  Air  Force  Base,  CBP  is  being  used  in  conjunction  with  a  decision  aid 
to  enhance  survivability  analysts'  preaictions  of  the  survivability  of 
structures  after  a  nuclear  blast. 


11.  BACKGPOUhD 

'.r'.  ‘‘iein  ;-ssr': '  •'‘■es ,  sp'-'-'serea  by  the  Air  Force  iJunian  Resources 

Laboratory,  Combat  Logistics  branen  (AFHRL/LRC) ,  conducted  a  study  to  assess 
tne  impact  of  psychological  stress  upon  aircraft  maintenance  task  times. 
Using  the  CBP  metnoo,  data  were  collected  from  52  expert  civilian  and  Air 
National  Guard  maintenance  personnel  to  predict  tlie  time  required  for  Air 
Force  aircraft  maintenance  personi'.el  to  perform  maintenance  tasks  in  a 
stressful  combat  scenario.  The  res-'lts  of  this  study  were  reported  by  Klein 
and  John  ( 1986) . 
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The  findings  of  that  report  revealed  that  the  subject-matter  experts 
(SMBs)  of  that  stuuy  predicted  aiicraft  miaintenance  time  for  Air  Force 
personnel  in  a  corhat  scenario  to  be  degraded  by  an  average  of  20%.  When  the 

tasks  themselves  were  examined  ii.oie  closely,  it  was  feuna  that  the  20% 

deceleration  in  aiicraft  maintenance  task  time  was  a  feature  of  complex 
tasks.  When  the  tasks  were  of  a  more  routine  or  simple  nature,  the  findings 
were  more  complicated.  Civilian  and  Air  National  Guard  i-MBs  did  not  agree 
about  predicted  maintenance  times  for  simple  tasks,  Civ/Iian  SMBs  predicted 
the  siHiple  tasks  to  show  no  deceleration  whereas  the  Air  National  Guard  SMBs 
predicted  a  40%  deceleration  in  aiicraft  maintenance  task  times. 

Til*-  divergence  of  predictions  between  simple  and  complex  tasks,  plus  the 

udditiunal  ditference  between  Air  National  Guard  and  civilian  SMBs'  predic¬ 

tions  lur  simple  tasks,  suggested  tliat  a  variety  of  factors  were  operating  in 
producing  predictions  of  maintenance  rime.  Knowing  that  e^^''''  expert  worked 
irom  his  own  unique  experience  base,  his  own  assumptions  abciit  the  combat 


scenario,  ana  his  assumptions  about  how  combat  conditioni  would  a  if  ter  from 
his  own  experience,  it  was  suspectea  that  the  causal  factors  that  produced  the 
preaictions  woula  uifier  across  experts,  'the  auait  trails  from  the  CbP 
process  confirmea  tins.  Tiie  experts'  assumptions  about  the  ooiiibat  scenario 
anu  how  that  aittered  from  their  own  experience  base,  incieea,  did  affect  the 
preaictions  they  maoe  about  tiie  effects  of  stress  upon  maintenance  time  in 
combat  situations. 

Having  learnec.  that,  even  in  this  small  database,  thi  relationship  between 
stress  ana  maintenance  time  was  a  r,:u It i faceted  one,  another  study  was 
undertuken  to  determine  the  nature  c>l  those  relat  ionsiiipr:  aim  coasLruct  i 
moael  that  would  aescribe  ana  predict  the  effects  of  stress  upon  aircraft 
maintenance  task  times. 


III.  BUILDING  THE  MODEL 


First  thoughts  abi^ut  a  mouei  to  describe  the  relationship  between  stress 
and  ma.ntainer  time  focused  upon  t.uilding  a  mathematical  model.  It  was 
anticij  atea  that  tiie  relevant  i.actors  would  be  identifiec,  weighted,  and 
combined  into  an  equation  that  wi-o  id  describe  and  predict  the  relat  ionsfiip 
between  stress  and  rr.aintenance  tin.e. 

WJjen  building  a  mathematical  ii  Kjel,  one  can  do  so  in  several  ways.  Ln 
empirical  approacti  may  oe  taken  in  wfiicfi  the  model  is  based  upon  one  data  set 
and  subsequently  tested  on  a  aitferent  c.ata  set.  This  approach  requires  large 
aataba..es,  especially  it  the  mouei  contains  more  than  twc  or  tiiree  1  actors. 

An-.ther  approach,  on  a  niore  conceptual  level,  is  appropriate  for  smiall 
uataba,;.es  sucfi  as  the  one  at  har.a.  This  approach  is  viai-le  wlien  there  are  a 
small  number  of  factors  and  when  tiie  relationship  araong  ilie  i  actors  is  fairly 
straignttorwara  (linear  as  opposed  to  ciirvilinear)  .  ll  i  ati  Leon  thought  that 
these  .issampt  ions  couro  be  so,  th<=  intention  was  to  examine  the  avaiiafile 

aata  anu  construct  a  simple  algeliraic  lormula  to  desciibe  and  predict 
per  for  ir.ance  time. 


A  aodel  was  envisioned  that  looked  something  like  the  fcllowing: 


Y  -  ax(l)  +  bx(2)  +  cx(3)  +  ax(l)  i  ex(:)), 


wf;ere  Y 
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maintenance  time,  -.’xpressed  as  a  percentage 
from  non-wartime  ana  non-stress  conditions, 
preparedness  for  the  hazard, 
ex[ ■  ;  1  ncfe  of  tec;  icians, 
complexity  of  tiie  rank. 


arm  so  on. 


increase  or  decrease 


Towaru  tins  end,  the  data 
factor  j  tfiut  were  reportea  to 


were  re-exanu  ncii  to  identi 
aftect  maintaincr  perfotn,.; 


a  pattern  for  tli<' 
b'  iimc. 
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Till  CbP  methoa  was  u-sea  in  the  earlier  stuoy  to  elicit,  expert  opinion  from 
the  SMi.s.  The  SMKs  haa  been  asked  to  aescribe  a  maintenance  task  thiey  had 
performed  in  wfiich  tfiere  hau  been  real  danger  to  themselvr-s  or  others.  Their 
task  haa  oeen  to  state  wnether  they  accomplished  the  task  more  rapialy  or  more 
slowly  in  the  emecgency  condition  (as  opposed  to  normal  circumstances)  ana  to 
aescrice  the  causal  factors  they  perceived  to  be  responsiole  for  that  tiire 
aiffertnce.  They  then  were  askea  to  consider  a  combat  scenario  and  to  make  a 
judgment  about  whether  maintenance  tiiiie  would  be  acceleraL.-a  or  decelerated  in 
that  Situation.  Again,  they  weti-r  called  upon  to  state  the  factors  that  th<y 
perceived  to  be  responsible  for  their  judgments. 

As  Will  be  reculira  irom  the  oii^jinal  study,  SMbs  cited  11  factors  that 
ditf erciitiateo  tm-  ..fiange  in  ma  intf-nance  Liime  rietween  their  own  experii  iice  ana 
the  coi.oat  scenar.io  lot  which  they  were  asked  to  rriake  a  prediction.  (Table  1 
depicts  these  causal  factors  ana  the  number  of  times  each  wr-s  cited.) 


Tei.'ie  ]  ■  Industry  Sf'.cs'  use  of  factors  Differentiating 
Alt  force  from  Industry 


Number  of  SMils 
citing  increment 
10 
9 


_ factor e _ 

Chunce  to  adapt  to  emergency  condition 
Nat  are  of  hazaru 

Tyi e  and  experience  af  technicians 

Consequence  of  uelays 

Iniormation  on  hazard 

Coi  piexity  of  tasK 

be  .'el  oi  Ccicroinat  itin  need •  a 

Precautionary  rr.easares 

Opportunity  to  seli-seiect  volunteers 
Nature  of  protective  equipnicnt 
Reaction  time  to  commuhicai  in.; 


Cuieful  scrutiny  of  tiie  rsi.n.al  factors  resulted  in  sevetal  observations, 
first,  tiie  factors  ate  not  inut.  t  ci .  .lent  oi  distinct  from  one  anothier.  Second, 
the  pri  sence  of  one  fac-foi  quite  often  implies  the  presence  of  another.  (For 
exainpl  ,  the  "nature  ot  nazaru"  would  dictate  whether  or  not  "precautionary 
m.easui  .s"  couia  or  .u.een.)  Thiio,  some  of  ttic-  factors  arc  very  general  in 
i.aturc  ("n.ituie  cl  hazard")  ana  others  are  more  specific  ( "oi  po  r  tun  i  ty  to 
self-s  feet  Volunteers"; . 


because  of  t;-e  a  soservat  ion.- ,  the  11  obtained  causa  i  factors  were 
coliap  .cd  into  four  lactors  to  be  a  clearer  rep  i  esentat  ion  of  the 

data.  'ine  tobusf  i  iture  ot  tiu  .  tactors  was  confirmed  .  ,y  '.xamining  how  tiie 
hMbs  u  eu  them,  asocrtaining  the  number  of  times  each  tac’tor  iiau  been  citeo, 
and  dt  eritiining  tiie  deceleration  in  mairi'.enance  time  attiibuten  to  eace 
1  actor  .  (See  Table  f  lor  the  c  ate  jor  ization  ot  causal  f  ictois,  the  frequent- 
ot  their  use,  and  the  decelerated  maintenance  time  attriauto.:  to  each.) 

At  till:,  point,  the  AfiiRL  contract  monitors  were  consultc-i  al'out  our 


preliminary  thinking  and  they  suggested  a  filth  factor  they  thought  to  l^e 
important  in  Describing  the  relationship  between  stress  nriO  maintenance 
performance. 

The  resulting  rive  factors  wer'-: 

-  preparedness  lor  tne  r.a'aro, 

-  taSK.  co;;,plexity, 

-  experience  of  tecfinicians, 

-  payoff  for  speec,  anc 

-  neeo  ror  otricts. 


Table  1.  causal  Factors  Di  t  fereritiating  Treaictec.  Air  Force  ana 
Inaustry  Maintenance  T  in.es  in  Lmergency  Coriuiti'.ns  Keportea 
by  Air  National  Guara  and  Ir.cnstry  Subject-Matter  Experts 


Air  National  Guara 


]  ndustr', 


Factor  Frequency  %  Decelerat  lor: _ F  actor _ Frequency  %  Deceleration 
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Thus,  the  causal  factors  luentified  by  the  CbP  pr  x  e.s; 
stuay  haa  oeen  reuucea  to  rive  critical  variables. 
resulting  five  factors  that  expert.:;  in  niaintenance  petfo’: 
aitect  i.iaintainer  performance  time.)  The  next  task  wao  ti 
building  blocks  in  such  a  v.iay  th-at  the  relationships  between  stress  and 
maintenance  tinie  could  be  aescribeu  ana  preaicted. 


Table  3.  Causal  Fa-; tor-;  i'.ittctinq  Aircraft  Maintenrince  Time- 
Causal  factor;. 
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is  suit  i'j  lently  conii^lex  tnat  even  r.i.e  most  experienceo  technician  will  tino  it 
so.  Tht retore,  the  relationship  between  (.xperience  ano  task  complexity  woula 
be  linear  to  the  point  at  whicli  the  task  is  so  complicatec  tnat  i*-.  makes  no 
airterence  how  experienceo  the  technician  is.  Thus,  what  naa  beer  a  linear 
relationship  woula  turn  into  a  curvilinear  one  when  extreiiiely  complex  tasks 
were  taken  into  account.) 

tqUuilly  important  to  the  lack  oi  inaepenaent  factors  is  the  issue  of 
interacting  variables.  Consiaer  tiit  situation  in  which  the  maintenance  task 
IS  a  simple  one.  It  is  reasonable  to  assume  that  simple  tasks  in  a  combat 
scenario  woula  be  pertorir.ea  faster  wnen  the  technician  coula  reduce  his  own 
exposure  to  danger  by  doing  so.  in  tne  other  iiand,  when  the  technician  would 
not  enhance  his  own  safety  by  hurrying,  there  is  no  reason  tia  preaict  that  the 
task  woula  be  performed  any  quicker  than  unuer  normal  c  ircurTiStances . 

The  original  aata  confirm  ttcso  expectat  ions.  V.nen  rraintenance  experts 
assumes  there  would  be  no  payoff  iccr  accelerating  maintenance  time,  they 
preaictea  simple  tasks  to  take  no  longer  in  a  combat  scen.jrio.  When  they 
assumea  there  was  a  payoff  for  t'.urryinc,  a  25%  acceleration  was  predicted. 

This  interaction  is  shown  in  Table  4. 


Taule  4.  Ine  Interaction  Between  Task  Complexity  and  Payoff 
tor  bpeea  in  Preaict ing  Aircraft  Maintenance  Time® 

Task  Po'.'ol  ■  Maintenance 

_ v-omprex  1  ty _ _ _ :or  S^.eea _ _ Acceleration 

sinpie  yes  +25% 

_ simple _ no _ 0% 

“Oiily  the  Eimpie  task  c<:na:..ion  is  odiown. 

The  comp  1 1C  a  tea  r.atuit  oi  •  iC  ..rtii  io  t  it  n  process  is  riOt  fully  captured  by 
the  prtceainy  exani/u.,  iiowevt.-.  .  -..bov..  reo  i-ct  ions  'wore  trade  by  experienced 

civili.-.ii  ma int. oiia- ice  pcir.o,,riei  ...:i  .iss'air.o-u  that  Air  Forcf-  rr.a intenance  personnel 
uiso  w.,ulo  oe  experiences.  on  ,Ai,i  '  j':  lonai  f'lU.ire;  Si's  made  his  predic¬ 

tions,  he  assuirieci  i  !i.-,  t  Air  Fore:-  •.aintainers  would  be  lut  xpi-r  lenced .  When 
this  experience  i  ,■  adoi  ’  the  relationship,  th.e  process  involves 

tiiree  ;  actors  an'i  i...  evo'n  nor-  a._/iirpl  icatea .  Table  5  uepicts  these  relation- 
snips.  Cnai, 0  11,0  t.he  a.t'- umpt  icn  oi  liavino  experienced  ter  tinic  ians  to  one  of 
riaving  inexper  icncea  tecnnicians  cnaisged  t  .e  .sign  of  the  p>redicted  maintenance 
timel)  Ail  of  i;.iie.se  fiouies  were  the  origina)  datr?.. 


ToLie  5.  lilt  1  :,tc  ra' ;  ,ons  Among  i.xper  ience  ,  rat’i  Complexity, 
ana  l-ayoft  ior  Speed  jn  1  rtaicLing  /-.ircraft  Maintenance  Task  Time 
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The  findings  in  Table  5  were  all  obtairiea  tor  the  condition  in  which 
others  were  not  neeaea  to  accomplish  the  tasK.  When  others  are  neeaed  to 
accomplish  a  simple  task,  an  even  more  complex  pattern  t-oierges  involving  four 
factors.  The  conaition  in  which  ochers  are  neeaed  to  complete  the  task 
proauceo  both  an  accelerated  and  a  decelerated  maintenance  time,  as  shown  in 
Table  6. 


Table  u.  Tiie  Interactions  An ang  Need  for  Others,  experience.  Task 
Complexity,  ana  Payoff  for  Speec.  in  Predicting  Aircraft  Maintenance  Time 
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Thus,  even  this  liiuted 
interactions  among  trie  factors. 


shows  the  importance  of  modelling  the 


If  a  mathematical  model  were  attempted,  these  interactions  would  have  to 
be  represented.  Interactions  can  ue  described  empirically  when  very  large 
databases  are  used.  It  is  even  possible  to  model  the  four-  and  five-way 
interactions  chat  would  be  requires  to  predict  the  effects  of  stress  upon 
maintenance  time,  however,  it  woi.ld  require  extremely  large  databases  to 
cuild  c^nu  subsequently  test  the  r.K  del,  Large-  databases  containing  this 
information  are  not  available  at  the  present  time.  Mathematical  modelling 
from  a  conceptual  standpoint  also  presents  serious  difficulties.  Five 
interacting  factorr.  strain  its  capacity. 

becau.se  of  the  problems  encoujiterec  ir;  liieeti.ig  tiic  a.Cjunpt  ions  of  a 
liiathematical  moaei,  an  alternativ.;,  form  of  modelling  was  sought  that  would 
capture  the  tuilnes.i  or  tiie  r erat  i.on.?.l,ip  between  stress  ana  maintenance  time. 

modelling  approacii  v/as  uesirea  tnat  would  preserve  the  power  of  using  expert 
judgmiciit  as  a  data  source,  wurie,  at  the  same  time,  be  sufficiently  rigorous 
to  adv.CiCe  our  knowledge  about  tiu.'  impact  of  stress  on  ma intt  lianre  times. 


JV.  THb  IDhNTiFlCATlON  POINT  MoLKL 

In  addition  to  the  aforecited  goals  oi  conceptual  rigor  r nd  utiJining 
expert  judcjnient,  the  modelling  ipprooch  had  to  produce  tool  t  l,at  would  be 
sufficiently  structured  to  allow  easy  application  by  thf-  user.  It  is  tnought 
that  tne  model  describeo  below  avccompl ishes  these  objectives.  however,  when 
reviewing  this  type  of  model,  the  reader  needs  to  depart  from  thinking  of 
equations  ana  formuitie.  Instead,  .i  framework  should  be  set  that  permits  tl;e 
uetiniLion  ot  variables  to  be  som.evhat  "fuzzy"  (as  detinod  try  the  context  in 
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which  tiiey  ate  embeadea)  while  still  being  uniform  enough  tc  maintain 
sufficient  rigor  and  to  have  meaning  for  users  of  the  mouei. 

The  tool  to  be  described  will  be  called  the  "Identification  Point  Model." 
The  laentif ication  Points  (IPs)  in  the  model  are  the  means  by  which  data  will 
be  storea  in  this  prediction  tool  and  also  the  anchor  points  at  which  the  end 
user  will  fina  answers  to  question;;  concerning  anticipated  changes  in 
maintenance  time  in  a  combat  situation. 

It  is  intenaea  that  this  luouei  will  allow  a  user  to  employ  each  factor  in 
the  moael  in  combination  with  every  other  factor.  One  level  of  each  of  the 
five  factors  ol  ttie  moael  will  niake  up  each  IP.  Thus,  each  IP  will  be 
uescribea  by  every  factor. 

When  using  the  moael,  the  users  woula  aefine  their  own  prediction  problem 
in  terinS  of  the  factors  in  the  moael.  For  example,  the  user  would  determine 
whether  the  task  was  high  or  low  on  complexity,  whether  the  need  for 
involvenient  of  others  would  be  hign  or  low,  whether  t)ie  payoff  for  speed  would 
be  hign  or  low,  ana  so  forth.  In  this  way,  he  or  she  would  match  the 
prediction  scenario  to  the  appropriate  IP.  After  locating  the  best-matched 
IP,  the  user  then  would  retrieve  the  empirical  evidence,  the  maintenance  time, 
whicfi  nad  been  previously  gathered  from  SMEs  and  had  been  storea  at  each  IP. 

Th>  type  of  moael  proposed  here  differs  from  a  matlieratical  formula  in 
several  ways.  The  IP  model  is  a  way  of  categorizing  existing  data  in  such  a 
form  tnat  the  data  can  be  easily  retrieved  by  both  sophisticated  and  novice 
users.  It  aifters  from  the  trauitronal  mathematical  model  in  that  the  problem 
aoes  nut  require  the  numerous  calculations  to  make  predictions.  It  does 
require  that  a  previously  acquirea  information  base  be  sequenced  and  stored  so 
as  to  serve  as  the  basis  of  estimating  the  effects  of  stress  upon  maintenance 
time.  The  tool,  itself,  woula  channel  the  user's  attention  to  the  most  nearly 
matcheu  IP  of  the  model,  bach  IP  would  contain  several  levels  of  detail  of 
information,  thus  giving  the  user  numerical  estimates  of  maintenance  time 
changes  ana,  very  importantly,  contextual  information  from  which  the  numbers 
in  the  nioaei  were  aerivec.  Thus,  the  user  would  not  be  relying  blindly  upon 
numeri  ;ai  estimates  alone. 

be  lore  going  to  an  actual  example  of  how  the  model  wouli;  be  used,  its 
constr.,ctior.  from  tne  original  auta  will  ue  aescribea. 

constructing  tne  IPs  from  the  Data 

The  first  step  in  constructing  this  moael  for  data  storage  ana  retrieval 
was  to  laentify  the  IPs  of  the  moael.  The  intent  of  the  mouei  is  that  each  IP 
will  represent  a  uiuque  combinution  of  factors  in  the  ncael.  That  is,  one  IP 
in  the  moael  wouj;  le  represented  by  a  high  level  oi  eacn  of  the  five 
factors.  Another  :P  would  repie:ient  a  low  level  of  the  first  factor  and  a 
nigh  level  on  each  oi  the  reri.aining  four  factors.  This  factorial  cycling 
would  continue  until  eacn  comoinat ion  of  factors  was  represented.  It  was 
decided  to  make  eac/i  factor  dichotomous  in  order  to  keep  the  moael  simple  anci 
to  make  the  clearest  referent  points  for  a  user  of  tht'  n.ouel. 
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The  re-examination  o£  the  data  haa  inaicated  that  five  factors  were  used 
by  the  i3^lhs  when  making  their  estin.ations  of  emergency  maintenance  time. 
However,  the  nature  of  the  hazara  had  been  assumed  by  all  the  SMEs  to  be  such 
tiiat  the  maintainer  could  prepare  tor  it  and/or  take  precautions  against  it. 

In  essence,  even  though  they  often  cited  the  nature  of  the  hazard  as  a  causal 
factor,  they  used  only  one  level  of  the  factor  as  defined  here  (that  level 
being  tne  case  when  precautions  could  be  taken) .  Because  the  SMEs  had  used 
only  one  level  of  this  factor,  the  preliminary  analyses  in  the  present  effort 
were  confined  to  the  other  four  factors.  When  the  two  levels  of  each  of  the 
four  remaining  factors  were  combined  (2  x  2  x  2  x  2) ,  a  matrix  containing  16 
It'S  was  obtainea.  Each  IP  in  the  matrix  was  assumed  to  be  in  the  condition  in 
which  iridintainers  coula  take  precauc ionary  measures  against  the  hazard.  The 
IPs  are  shown  in  Table  7. 


Table  7.  The  laentif ication  Point  Model 


Numerical  Classification  of  laent if ication  Points 
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tour  of  the  factors  of  the  inoael  are  listea;  task  complexity,  experience 
of  the  technicians,  payoff  for  speed,  ana  need  for  others.  "H"  stands  for  the 
high  itvel  of  a  factor.  "L"  stands  for  a  low  level  of  a  factor.  The  level  of 
each  oi  the  factors  at  each  of  the  16  IPs  is  defined  by  the  column  under  the 
IP  number.  For  example,  IP  #7  i.s  characterized  by  low  task  complexity,  high 
experience,  low  payoff  for  speed,  and  high  need  for  others.  All  data  that 
were  collectea  in  which  OMEs  assumed  this  unique  combination  of  factors  to  be 
operative  are  representea  at  this  IP. 

In  the  origi:;,.  L  .■,t  j.;y  Sfilin  -uimed  that  precautionary  measures  could  be 


taken  lor  the  haiirc;  .so,  this  I.,-, 
will  i  itroauce  ti.  ..tne!  level 
coula  .ot  prepar*;  for  me  n^zarc, 
this  .s  .  udy . 


tor  was  not  dichotomized  in  this  Table.  We 
this  factor,  that  being  when  the  technician 
when  we  discuss  additional  data  collected  in 


rtssign.ng  Experts'  Judgments  to  t)ie  IPs  of  the  Model 


Til"  data  were  examined  to  determine  the  best  match  h'-tw-t-en  the  IPs  and 
each  c  ise  outainea  in  the  study.  q  case  is  defined  tiere  as  <in  S.'-'.E '  s  judgment 
ot  mai  .tenance  tin-.e  either  for  his  own  tasK  as  recounted  to  the  interviewers 
or  for  the  iiypotlietical  combat  scen-ario.  A  Crireful  examination  of  all 
record  .a  responses  ot  each  SHE  was  made  in  order  to  determine  what  that 
p-ersor. '&  assumptions  were  about  each  ot  the  factors  in  tiie  model.  After  the 
ueterii.  .nation  was  made  of  whettier  each  case  was  high  or  low  on  each  of  the 
ractors,  the  case  was  ascribed  to  the  appropriate  IP. 


Several  analyses  were  conaacted.  The  civilian  SMKs'  estimations  of  the 
change  in  maintenance  time  between  their  own  tasks  performea  in  an  emergency 
situation  ana  the  same  tasks  performed  under  normal  circuinstances  were 
analyzea  (Table  8) .  The  predictions  of  these  same  experts  for  a  combat 
scenario  were  analyzed  (Table  9).  The  predictions  of  Air  National  Guard 
personnel  were  tabulatea  (Table  10) .  Finally,  all  the  data  were  combined 
(Table  li)  to  obtain  a  first  look  at  how  maintenance  time  was  distributed 
across  the  aifferent  IPs  in  the  model. 

The  IPs  in  Table  11  contain  judgments  from  varying  numbers  of  SMEs, 
ranging  from  one  per  IP  to  six  pet  IP.  Several  experts'  scores  were 
eliminatea  (one  score  from  SME  5,  one  from  SME  12,  and  one  from  SME  16) 
because  they  coula  not  be  categorized  along  all  tour  factors  with  any  degree 
of  certainty.  Two  oata  elements  were  changea  from  the  way  they  had  been 
erroneously  scorea  in  the  first  report  (SME  2's  description  of  differences 
between  normal  ana  emergency  conditions  ana  SME  4's  Air  Force  prediction). 

Two  data  elements  were  adaed  because  two  experts  (SME  6  and  SME  18) 
distinguished  between  experiencea  ana  nonexper lenced  personnel,  reporting 
corresponding  numerical  estimates  of  maintenance  time. 

Table  11  allows  a  first  look  at  how  well  the  model  organizes  the  data  and 
a  preview  of  its  potential  predictive  power.  Eleven  of  the  16  IPs  in  Table  11 
contain  data.  The  vestiges  of  the  findings  of  the  earlier  report  can  be 
seen.  At  IP  8  where  the  task  is  simple,  is  performed  by  experienced 
technicians,  has  a  low  payoff  for  speed,  and  does  not  need  others  to  perform 
the  tasK,  no  change  in  maintenance  time  is  seen.  This  corresponds  to  the 
first  report.  At  IP  5,  where  the  task  is  complex,  is  performed  by  experienced 
technicians,  has  a  low  payoff  for  speed,  and  needs  others  to  accomplish  the 
task,  a  median  value  of  20%  performance  deceleration  was  obtained.  Again, 
this  finding  corresponds  to  ttie  results  obtained  in  the  original  analysis. 

The  other  IPs  are  even  more  interesting  in  that  they  contain  what  were 
formerly  the  extremes  in  the  range  of  obtained  findings  in  the  preliminary 
study.  The  absolute  values  fluctuated  as  would  be  expected  in  this  type  of 
effort  (especially  since  the  IP  framework  was  imposed  after  the  fact),  but 
prediction  values  of  the  same  sign  reside  within  individual  IPs.  This  is  seen 
at  IPs  1,  3,  4,  8,  and  12,  wheir.  the  signs  of  the  estimated  changes  in 
niainter.ance  time  are  tiie  same  within  each  IP.  IP  5  contains  one  0  and  four 
oecelei dtiori  estimations.  IP  7  holds  two  Os  and  four  deceleration  values. 

The  tact  that  the  "extremes"  fail  into  nieaningful  patterns  witiiin  the  IPs 
ienus  substantial  credence  to  the  utility  ot  this  mocel. 

Looking  at  the  way  the  niouel  orders  the  data,  it  is  seen  that  a  great  ueal 
of  information  is  at  hand  to  predict  maintenance  time  on  tasks  performed  by 
technicians  witn  appropriate  (type  ana  amount)  experienci .  This  is  likely  due 
to  the  tact  that  the  SMES  who  worked  through  the  CBP  method  to  produce  these 
data  were  experienced  technicians.  Six  of  the  eight  IPs  in  "experienced 
technicians"  condition  contain  data. 
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Table  8.  Change  in  Maintenance  Time  Between  Normal 
and  Emergency  Conditions  in  Industry 


hiqm 

Exper lence 
TecMniciene 

IP 


Low 

Experience 

TecMniclene 

IP 


High  peyoff  for  speed  Low  peyo^^  for  speed 

High  complexity  Low  complexity  High  complexity  Low  complexity 


high 

low 

high 

low 

high 

low 

high 

1  ow 

need 

need 

need 

need 

need 

need 

need 

need 

for 

for 

for 

for 

for 

for 

for 

for 

otHere 

othere 

others 

others 

others 

others  othere  othere 
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0 

n 

0 

0 

-20 

0 

0 

-20 

-19 

0 

-100 

-153 

1 

2 

3 

4 

5 

6 

7 

8 

+  =  performed  faster  than  normal. 
-  =  performed  slower  than  normal. 


o  o  o 


Table  9.  Change  in  Maintenance  Time:  Industry  EME's  Predictions 
of  Mr  Force  Combat  Maintenance  Time 


High  payoff  for  apeed 


High  complexity  Low  complexity 


high  lox  high  lo^ 

need  need  need  need 

for  for  for  for 

othore  othere  othsre  othere 


Low  poyoff  for  speed 


High  complexity  Low  complexity 


high  low  high  low 

need  need  need  need 

for  for  for  for 

othere  others  others  othere 


High 

Cxper 1 encs 
Techniciens 


Low 

Experience 

Techniciena 


+  =  perfi'rmed  faster  than  normal. 
-  =  performed  slower  than  normal. 


Table  10.  Change  in  Maintenance  Time:  Air  National  Guard  s 
Predictions  of  Ait  Force  Combat  Maintenance  Time 


High  payoff  for  apaed 


Low  payoff  for  apaed 


High  complexity  Low  complexity 


High  complexity  Low  complexity 


high  low  high  low 

need  need  need  need 

for  for  for  for 

othera  othera  othera  others 


high  low  high  low 

need  need  need  need 

for  for  for  for 

othera  othera  othera  othera 


High 

Exper 1 ence 
Tachni c 1  ana 


Low 

exp  ar 1 anca 
Technic  1 an« 


+  =  performed  faster  than  normal. 
-  =  performed  slower  than  normal. 
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Exper 1 ence 
Tecr*n  i  c  l  ana 

IP 


Low 

Expar i anca 
Tac^>o  t  c  1  ana 

IP 


Ijble  II.  Change  in  Maintenance  Time:  All  SMEs'  Predictions 


Hign  payoff  for  apeed 


HIQM  complexity  Low  complexity 


Low  payoff  for  apeed 


High  complexity  Low  complexity 


h  1  gh 
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other  a 

otnore 

-11 

+33 
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0 

0 

0 

-19 

+  74 

+25 
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0 
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-20 

-13 

0 

-33 

-19 

0 

-iOO 

-19 
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-153 

1 

2 

3 

5 

6 

7 

8 

o 

CM 

1 

-22 

-40 

-50 

-67 

-50 

-66 

9 

10 

1  1 

12 

13 

14 

15 

16 

performed  faster 
-  performed  slower 


than  normal, 
than  noripa  1 . 
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within  ti>e  "exper  ienceo.  technicians"  conuition,  i*.  ^-ari  bf  r.een  that  wh.en 
there  is  a  pa^oit  tor  spteo  in  completing  the  task,  tasks  (IP5;  3  ana  4) 

are  preuicteu  to  be  performed  more  quickly.  This  holu.-.  true  whether  the  task 
lb  performeu  alone  or  others  are  needed  to  complete  it,  i.hen  the  task  is 
complex  ana  others  are  neeaed  to  con'.plete  it  (IP  1),  maintenance  time  is 
judged  to  be  slowed  (19%),  even  tiiough  there  is  a  payoff  Lor  speed  in 
accomplishing  the  task. 

Stiii  focusing  upon  exi-e  r  icnced  tcchn  icians,  v/hen  there  is  no  payoff  for 
speeu,  .'-imple  tasks  are  preaieti-o  to  take  neither  less  nor  more  time  when 
performed  alone  in  a  combat  situation  (IP  t  is  uniformly  0).  When  others  ar‘t 
neeoeu  to  complete  a  simple  r.c.sk  tor  wrocn  there  i"  low  payoff  for  spoeo  (IP 
1),  miore  time  is  needed;  tne  .'.ittur  noLcs  o  r  a  complex  task  witri  trie  same 
parameters  (IP  b;  .  iBotri  ci.'irrdex  ana  simple  tasks,  IPs  b  ana  7,  recei'.t  a  a 
niiQ-range  value  ot  nearly  2Ci  loneer  tiiV;e  t-'  ao  the  task  when  there  was  no 
payofi  ror  speed.) 

oiven  even  tnis  preliminary  analysis,  it  cari  be  saic  that  these  Lour 
factors  interact  to  produce  distinctive  combat  c  ircum.-.tances  in  wriicn 
miainte r.ance  tiiue  can  be  preaicteu.  Maintainer  per foriuance  tim.e  wcuiu  be 
oeceiuratea  ir.  tne  c  loumstani.os  oosci ,  iied  iiy  iPs  1,  7,  :u,  ]i,  1,^,  13,  ana 

lo.  It  woulu  be  aoceitratea  in  situations  simiiar  to  rhoi.e  at  IP;-  3  ana  4.  It 
would  snow  no  Change  in  a  scenario  such  as  IP  8. 


V.  HOw  'r-iK  MOCFL  WOUID  BE  USob 

The  first  step  in  utilizing  the-  mcxlol  would  be  tor  the  user  to  niatch  the 
preaicLion  situation  to  the  appro;.d  late  IP  m  the  nicxioi.  In  order  to  ao  this, 
the  prediction  situation  itseri  would  be  categorized  by  simply  aetermining 
whether  it  was  high  or  low  on  eacli  of  the  four  fa-tors.  (The  re-^d^r  may  wish 
to  return  to  Table  7  to  see  the  If  model.) 

for  example,  11  the  n  1  tua-  m  question  were  higt,  on  all  the  factors  in 
Table  7,  IP  number  1  wouia  be  the  host,  matcti.  It  ti^e,  prediction  scenario  were 
nigh  Oh  experience  01  technicians  -n.nd  aiE.o  liad  a  high  payoff  tor  speed,  but 
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>.  t  ineaicJi'j  Vi.; 

lue  ot  pertormance 

dectlerai 

iun/aco 

t:  le rat  ion 

couia  ie  oibtci  i neoi .  A  singfe  vai'ue,  or  a  luear.  accompani-jo  by  a  slanuar-J 
deviation,  wouta  be-  usoLui  wfien  the  user  wanted  only  a  g<-neral  estimate  or 
wanted  to  combine  bt-veraL  single  values  into  a  planning  exercise  tiiat  requiieu 
estimates  LOT  multiple  CaSGS. 


On  a  more  uetailed  level,  the  entire  ranye  ol  valuef.  aL.alo  to  obtained, 
tnus  yiviny  more  iniormation  to  the  user  and  ailowiny  liii.-  n.'-er  Lf)  see  tlic 
variation  in  the  data,  the  aistr ibution,  the  worst  case,  i he  best  case,  and 
tne  moael  case. 


Finally,  the  user  coula  request  the  actual  case  uescriptions  (the  task, 
the  maintainer,  the  context  in  which  the  task  was  periorr.itja,  etc.)  from  which 
tne  numerical  uata  were  aerivea.  Tnis  woula  allow  the  user  to  gain  even  more 
insight  into  the  phenomenon.  Ttsereby,  a  better  unaerstanu  ii'.y  of  the  causal 
factors  that  arove  the  predictions  would  be  obtainea.  Special  causes  that 
might  operate,  eitfier  in  tiie  particular  case  the  user  is  estimating  or  in 
analog  v^ases,  coula  be  determinec  i rom  rnis  use  of  the  model.  This  woula  be 
especially  helpful  it  the  user  were  going  to  employ  the  nioael  to  engage  in  a 
CBP  process.  In  such  an  instance,  tlie  most  closely  matched  r,-;sc-(s)  at  or 
clustered  around  the  IP  coula  serve  as  the  comparison  case  in  a  CBP 
application. 


As  an  example  of  how  this  tool  might  be  utilized, 


bxaniple  1  is  a  simpio  one  that  requires  others  in  oraer  to  get  it  done. 
(Table  i2  aescribes  tne  situation  of  Kxampie  1.) 


Table  12.  Exaniple  1 

This  situation  woula  be  character izca  oy ; 


I'ask  Complexity 
Experience  of  Techs 
Payoff  for  Speeo 
Neea  ii.a  Others 


This  pattern  of  situation  characteristics  matches  IP  7  ut  the  model.  The 
change  in  miaintenance  time  aatt.  at  IP  7  can  be  retrievea.  The  aata  may  be 
used  iti  summary  lorm,  aii  percentage  figures  may  be  retrievea,  or  the 
vignett.es  uescribing  the  oases  from  which  the  percentage  of  changes  in 
maintenance  tiiiie  were  aerivea  may  oe  retrieved. 


Data  front  IP  7  are  shown  in  Table  13  and  Appendix  A. 


Table  13.  Information  About  Aircraft  Maiitenance 
Time  Retrievea  irom  IP  7  for  Example  1 


Meuian  -  19%  acceleration. 
Range  =  0%  -  i53%  aeceieration. 


Aii  obtairiec  percentages: 

0,  0,  13,  19,  19,  153%  deceleration. 


user  is 

'  1  n 

by 

h  in 

• 

VI.  ADDITIONAL  DATA  COLLECTION 


I  Aaaitionai  aata  were  coliectea  in  Lne  summer  o£  I^ob  in  ^ri  attempt  to 

»  outain  information  about  several  of  the  IPs  for  which  data  were  unavailable, 

f  It  was  hopeu  also  to  yet  a  preliminary  notion  of  whether  inforniation  coliectea 

irom  maintainers  performing  in  a  oifferent  environment  woula  be  similar  to 
'  that  ootainea  in  tne  preliminary  study. 

I  Date:  wete  gutherea  from  SMEs  respotisible  for  the  maititenance  of  tire- 

'  fighting  equipnient  one  who  hau  recently  used  it  to  extinguish  a  fire  resulting 

I  from  a  large  chemical  spill.  This  was  an  extremely  dangerous  situation  for 

■  the  personnel  wrio  tougnt  the  iirt-  ano  lor  the  maintenance  people  who  p>r  f  oriT;'/c 

i  tneir  repair  tuOKs  at  the  sit*;  of  the  fire.  Four  days  were  neec.cd  to 

extinguish  the  file.  Toxic  guses  releasea  Ltom  the  tire  foriata  a  clouc  that 
tnreattneu  aagacent  communities,  several  oi  which  haa  to  he  evacuated  during 
the  course  of  the  inciaent.  Data  were  collected  appruxin.ately  2  months  after 
tne  inciuent  occur rea. 

Only  two  rriu intenance  people  were  involved  in  the  inciaent.  Tne  data  po<  l 
IS  therefore  .sir,aii--similar  in  number  to  the  Air  National  Guara  ."^MEs  (n  =  3) 
in  tne  Lirst  stuuy.  however,  it  must  be  remembered  that  the  intent  of  nsinu 
expert  juagii.ent  is  net  to  run  large  numbers  of  subjects  as  in  an  experiment, 
tut  to  capture  critical  information  within  the  experience  base  of  each  expert. 

Tne  CbP  m.etnoo  was  usea  to  leaa  the  SMEs  to  a  preoiction  of  n'.aintenance 
lime  change  in  an  Air  Force  combat  scenario.  The  two  SMi.s  wt-re  ma i ntenat'ice 
pecsonr.ei  pern.anentiy  employeo  by  an  urban  fire  uepac  ti;.e..t .  one  was  a  highly 
txperienceu  maintenance  person  who  also  haa  some  experience  cealing  with 
emerger.cy  situat.  ,  althouga  t;  •  .'-.e  emergencies  had  poseu  no  threat  to  his 
own  personal  Sai-.i.  ..  'r.ne  otiiui  oMi,  was  less  e  ■•.per  lencec  in  n  a i ntenance  ( r.e 
calieo  niniself  rateiy  eX;  ''.-eo;,  out  did  liave  oxpecie-nce  it: 

1 1  r  el  igf.  t  ir.g  ,  w.  .  .  ti.^ails  a  .oerabTe  aniount  of  expt'sure  to  perse, nal 

uanger . 

The  SI-lLs  were  guicieu  ttirougii  tne  CBP  methixi  on  two  separate  occasions, 
oeven  LuSKS  were  tecountea  ano  seven  preuictions  maae.  iour  tasxs  were- 
uescriieu  by  the  less  experiencea  niairitainer  anu  three  b\  thie-  iiighly 
txperiL-nceu  niaintenance  person. 

Tti-'  Sl-iLb  first  were  askeu  to  uetermino  ti;e  differenc.-  in  hir,,  hntwi  eri  t.iie 
normal  anu  tlie  eniergency  conaitions  to  cortiplete  their  '.wn  tasK.s.  A 
nypotheticai  Air  Force  scenario  wa.s  then  oescribed  and  trie  experts  were  askea 
to  preoict  the  time  for  Air  Force  n.aintenance  personnel  to  complete  a  similar 
task  unuer  emergency  conditions. 

Eacti  of  the  tasks  perforri.ed  in  this  emergency  resulting  from  tht^  spill  war 
evaluatea  separately  on  each  of  the  four  factors  in  the  niodel  in  oruer  to 
uetermine  the  IF  which  it  sh-  i  lu  be  assigned.  The  same  proceaure  was 
followea  for  eaci  ^  reaiction  get. '.  rated  for  the  Air  Force  scenario. 


Tai  Les  14  ana  lb  la-’  i4  .  i  shicawii 
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be  seen,  a  lemarkable  ct,  t  ireCi- ■  .t:(:  \  n  i  w.  ^  aic  w,-..:  ctained. 
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were  uti  o  oecuaSe  ttie  vil-ii  r  n 
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Table  16.  Prepared  Condition 
Change  in  Maintenance  Time:  All  SMEs  in  Both  Studies 
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+  =  performed  faster  than  normal. 
-  =  performed  slower  than  normal. 


Table  17.  Unprepared  Condition 
Change  in  Maintenance  Time  Between  Normal  and  Emergency  Conditions 
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teatuie,  a  case  description  of  each  incident  that  servc-c  as  the  comparison 
case  also  can  be  gleaned  from  this  type  of  modeling,  thereby  providing  users 
with  wackground  information  from  which  the  maintenance  deceleration  numbers 
were  uerived. 


VIII.  RECOMMENDATIONS 

In  further  refining  this  model  to  preaict  the  effect  of  stress  upon 
maintainer  performance  time,  what  should  the  next  steps  be? 

1.  Full-scale  aata  collection  should  occur. 

z.  An  automated  decision  aia  shoula  be  developed. 

3.  The  preaiction  values  obtainea  from  the  model  should  be  synthesized 
with  extant  information  about  sortie  rates,  thereby  enhancing  the  quality  of 
the  predictions  on  sortie  rate  and  answering  the  overriaing  questions  on  this 
issue  posed  to  the  planner. 

A  full-scale  study  shoulo  be  conductea  to  obtain  the  necessary  information 
to  fill  each  of  the  IPs  of  the  model.  Ten  to  fifteen  data  points  would  be 
collectea  for  each  IP  of  interest,  thereby  generating  sufficient  numbers  from 
which  to  construct  estimations  of  maintenance  time. 

In  a  full-scale  study,  the  boundary  conditions  of  the  Air  Force  combat 
scenario  woula  be  specif lec  for  the  CBP  method  to  be  usea.  For  example: 

SMEs,  when  predicting  the  Air  Force  maintenance  time,  would  be  told  that  the 
technicians  in  that  situation  would  not  be  highly  experienced.  They  would  be 
told  to  make  their  precictions  about  a  complex  task  that  was  performed  alone. 
The  aitount  of  preparation  that  would  be  expected  for  a  particular  type  of 
combat  situation  woula  be  specified. 

in  addition  to  the  relevant  causal  factors  derived  from  the  present  cata 
base,  otiier  factors  shoula  be  incluaea  as  well.  The  role  of  sleep  deprivation 
and  fatigue  woula  be  systematically  investigatea.  Very  importantly,  the  issue 
of  ttie  quality  of  performance  shoula  be  adaressed  since  a  task  that  is  done 
incorrectly  has  a  critical  impact  upon  the  aircraft's  performance  in  combat. 

'Inere  are  groups  of  non-miiitary  maintenance  personnel  whose  job  it  is  to 
spearheaa  the  cic  mup  work  for  chemical  spills  and  other  similar  incidents. 
(Such  personnel  i.uu  tne  task  ui  cutting  open  the  tanker  carrying  the 
phosphorous  in  tne  inciaent  aescribed  in  the  secona  phase  of  this  paper.) 

They  work  in  extremely  hazaraous  situations,  each  of  which  nay  be  somewhat 
unique.  Consequently,  preparatijn  cannot  be  as  extensive  for  them  as  for  the 
SMEs  (constituting  tiie  current  aatabase)  in  the  hazaraous  chemical  plants. 

The  unutrstanaing  of  how  tfie  lacit  of  preparation  affects  the  performance  of 
technical  tasks  in  stressful  conaitions  woula  be  enhanced  by  using  such  a 
grou[.  of  SMEs.  Again,  the  non-military  judgments  woulc.  be  augmented  hy  those 
from  Air  Force  personnel,  especially  with  respect  to  quality  of  performance. 


Since  the  bounaary  conaitions  of  the  IPs  in  the  moael  can  be  specified 
betorenand,  the  oraer  in  which  to  collect  data  for  the  IPs  is  not  rigidly 
uefineo.  Consequently,  special  situations  (IPs)  that  are  of  greatest  concern 
to  the  users  couio  be  isolated  ana  given  heavy  emphasis  when  collecting  data. 
This  couio  be  done  at  any  stage  in  the  research. 

The  IP  model  outlined  in  the  previous  pages  is  derived,  methodologically, 
from  a  completely  crossed  factorial  design.  However,  it  would  not  be 
unexpected  that  certain  cells  in  such  a  design  would  have  little  meaning  for 
nir  Force  planners.  If  this  were  the  case,  no  effort  need  be  wasted  in 
collecting  data  for  sucn  conaitions. 

An  automated  decision  aid  could  be  developed.  Using  this  decision  aid 
would  ue  easy  ana  the  database  itself  would  be  tied  to  a  user's  computerized 
information  system.  The  decision  aid  would  be  one  that  users  could  employ 
while  aitt-.^g  in  their  own  offices.  In  such  an  application,  the  user  would 
enter  into  the  computer  pertinent  information  about  the  situation  for  which 
predictions  were  needed  on  each  of  the  model's  dimensions.  (For  example, 
would  tha  technicians  be  experienced?  Would  the  task  be  a  complex  one?)  Tne 
computer  would  reply  with  the  predicted  change  in  maintenance  time  for  the 
type  or  scenario  entered  by  the  user. 

In  addition,  the  cases  from  which  the  predictions  were  derived  could  also 
ue  preaentea.  This  would  be  useful  if  the  user  wanted  to  follow  the 
predictions  process  employed  by  the  SMHs  from  whom  the  data  were  collected,  in 
order  to  further  refine  his  or  her  own  predictions.  The  information  obtained 
from  tne  prediction  model  could  be  synthesized  with  other  Air  Force  databases 
concerning  sortie  rates. 
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APPENDIX  A;  CASE  DESCRIPTIONS  OF  AIRCRAFT  ^U^INTE^1ANCE 
TIMES  RETRIEVED  FROM  IP  7  FOR  EXA^iPLE  1 


0%  (No  change) 

This  SME  haa  been  a  chemical  plant  operator  for  over  5  years.  The 
gob  he  selectea  as  a  comparison  case  was  taking  samples,  which  usually 
requires  20  minutes.  In  this  case,  he  was  taking  saii;ples  of  a  chemical 
agent,  ana  in  the  process,  the  fluia  ran  over  his  gloves  and  portions  of 
his  suit.  The  gob  still  took  20  minutes  per  sample.  He  did  not  see  any 
stress  etfect,  given  his  level  of  experience. 

0%  (No  change) 

This  SFiE  was  a  chemical  plant  operator.  The  task  he  usea  as  a 
coiuparison  case  was  cleaning  a  leak  tiiat  developea  alter  a  machine  broke. 
The  machine  was  full  of  a  liquia  which  had  leaked.  Ihe  cleanup  process 
took  2  hours.  No  effect  of  psycliological  stress  was  reportea. 

19%  deceleration 

This  SME  worked  as  a  chemical  plant  operator.  Viis  auties  were  to 
aestroy  the  chemicals.  These  chemicals  would  arrive  in  various  types  of 
containers.  He  would  unpack  them,  remove  the  chemical  by  shearing  or 
drilling  into  its  container  and  draining  the  chemical,  and  incinerate  the 
container  in  a  furnace.  To  perform  this  job,  he  would  enter  a  specially 
sealed  room.  He  would  also  repairs  on  equipment  in  the  room,  connect 

hoses,  and  so  forth.  The  average  worker  entered  the  room  50  to  100  times 
auring  their  employment.  There  was  a  2-hour  limit  to  the  time  a  worker 
coula  spend  in  the  chamber,  and  a  limit  of  two  sessions  per  day.  Over 
6,  000  total  entries  to  the  room  had  been  .made  without  aaverse  effects. 
Operators  wore  the  DPE  ensemble  describea  in  Klein  ana  John  (1986). 

The  task  performea  in  tne  inciaent  he  described  was  replacing  a 
connecting  line  lor  chemicals.  Connecting  lines  wear  out  and  beccne 
blocked.  The  task  was  a  routine  one.  Perhaps  the  b'.'St  corriparison  was  to 
performing  hydraulic  system  repairs  with  the  pressure  oft.  Ordinarily  he 
could  replace  a  100-foot  section  of  line  in  4  hours.  Wearing  the  special 
suit.  It  would  take  him  4  hours  to  replace  50  feet  or  line.  In  this  case, 
the  line  had  been  "eaten  out"  oecause  of  the  chemical.  The  pumps  iiaa  to 
Ofc  turned  oft,  and  valves  closed.  There  were  chendculs  in  the  linr  anu  on 
tfi'i  floor.  Tne  task  required  team  coordination.  Th.  y  couiu  see  locks. 
Th'jy  could  see  chemical  vapor  in  the  air.  Under  the..e  conoit  i^us,  1  r  a 
50-toot  section  that  they  could  usually  replace  in  4  hours,  the  tusK  took 
them  (ana  the  follow-on  teams)  an  additional  42  minuses.  The  extra  tinie 
Was  needed  in  part  because  of  the  restrictions  caused  by  the  suits  triey 
were  wearing.  Tney  could  not  work  too  fast  for  fear  that  they  would  tear 
tr.e  suits.  The  suits  were  cumbersome  ano  heavy,  alti.ough  cornu, unications 
.ere  not  a  prouiem.  In  part,  the  extra  time  was  due  to  physical  stressors 
such  as  heat  ana  ljumiaity.  A  magor  reason  for  the  time  increment  was  the 
eiTiOtional  stresses.  Personnel  were  more  distracted  by  noises  and  sudden 
movements.  Tney  checked  on  their  buddies  more  often.  They  checked  their 
breathing  apparatus  more  often.  They  got  tired  faster  and  had  some 
difficulty  concentrating  on  the  tasks.  They  checked  and  double— chocked 
the  subtasks  more  carefully.  In  general,  they  tenaeu  to  move  more  slowly 
and  carefully. 


153%  deceleration 

This  &ME  had  workea  as  a  chemical  plant  operator  for  3  years.  The 
task  he  selected  as  the  comparison  case  was  hooking  up  different 
containers,  a  15-minute  task.  With  chemicals  in  the  containers,  the  task 
tooK  between  35  and  40  minutes,  primarily  because  he  kept  double-checking 
to  make  sure  he  had  completed  each  step  and  that  no  connections  were 
leaking.  He  deliberately  moved  more  slowly  and  carefully. 


